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Background 


Attempt to conceptualize and extend lessons 
learned from experience with NASA research 
and development programs and projects. 


See Walker AIAA 2011-3345 
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Motivation/Technical Challenge 


Development of confidence in design and analysis 
with limited resources. 
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Proposed Solution 


Higher fidelity, integrated uncertainty quantification 
performed earlier in the design process should lead to: 

* Better understanding of inherent risk 

* Ability to better direct resources 

■ More clarity between conducting further test and analysis 
versus redesign 

■ Appropriate selection of model fidelity for applicable design 
maturity state 

* More confidence in design and analysis 
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Assessment Space 


A 



Model Fidelity 


* Relates program/ 
project/design 
complexity and maturity 
to its representation 
(model fidelity) 

* Costs typically increase 
away from the origin 

* As the system matures, 
the costs of determining 
a problem and fixing it 
increase rapidly 

* Axes have realistic limits 

* Confidence: 4 th axis 
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Complexity 


A 

Architecture 
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Maturity 


NASA Project Life-Cycle Phases 

NASA Systems Engineering Handbook 
NASA/SP-2007-6105 Rev 1 



Phase F: 

Closeout 

Phase E: 

Operations and Sustainment 


N/ Phase D: 

System Assembly, Integration and Test, Launch 


Phase C: 

Final Design and Fabrication 


Phase B: 

Preliminary Design and Technology Completion 


Phase A: 

Concept and Technology Development 


Pre-Phase A: 
Concept Studies 
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Model Fidelity 


* Computational aerodynamics example 

* Similar example could be constructed 
with test and evaluation or other 
disciplines 
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Model Fidelity/Uncertainty Models 



Model Fidelity 


Pate-Cornell, M.E. (1996) “Uncertainties in Risk Analysis: Six Levels 
of Treatment.” Reliability Engineering and System Safety. 54:95-111. 
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Mapping the NASA Design Life-Cycle 
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Mapping the NASA Design Life-Cycle 


Reviews 

• MCR - Mission Concept 

• SRR - System 
Requirements 

• PDR - Preliminary Design 

• CDR - Critical Design 

• TRR- Test Readiness 
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Concept and Technology Development 
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Phase B: 

Preliminary Design and Technology Completion 


PDR 
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Mapping the Design Cycle 
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Mapping the NASA Design Life-Cycle 


Reviews 

• MCR - Mission Concept 

• SRR - System 
Requirements 

• PDR - Preliminary Design 

• CDR - Critical Design 

• TRR- Test Readiness 
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Uncertainty Quantification 
in the Design Cycle 


Phase A: SRR 



• Typically discipline based 

• Often enters in Phase B 
(before PDR) 

• Early assessments tend to 
be engineering judgment 
based 

• Ad-hoc 

• Key component of risk 


More representative uncertainty models Better informed risk assessments 
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Uncertainty Classification 


Aleatory Uncertainty is inherent variation associated with a parameter, 
physical system, or environment 

•Also referred to as variability, stochastic uncertainty, irreducible uncertainty 

• Is a characteristic of the system being analyzed 

• Resources directed to robust design 

• Examples: 

■Variability in geometric parameters due to manufacturing 

■ Material properties, weather conditions 

Epistemic Uncertainty arises from imperfect knowledge or ignorance 

•Also referred to as subjective uncertainty, reducible uncertainty, or model-form 
uncertainty 

• Is a characteristic of the state of knowledge of the analysis team 

• Resources directed to better understanding the problem 

• Examples 

■ Insufficient experimental data to precisely characterize a probability distribution 

■ Poor or limited understanding of physics phenomena or physics coupling 
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Uncertainty Integration 


Phase A: SRR 



• Originates in discipline 

• Critical means of 
communication in the 
discipline interface 

• Development of discipline 
interface with appropriate 
exchanges of information 
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P-box Representation 
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P-box Representation 



Sample Value 
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P-box Representation 



Sample Value 
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Management of Risk and Resources 


Passes 

Verification 


Red Box indicates a 
“keep-out” zone 
defined by the 
probability and 
response requirements 

Margin is the minimum 
horizontal distance 
between the p-box and 
the keep-out zone. 

The p-box can also be 
represented with a 
complementary 
cumulative distribution 
(1-CDF) 
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Management of Risk and Resources 


Verification 


P-box crosses into the 
keep-out zone 

Indicative of possible 
negative margin in the 
system 
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Management of Risk and Resources 


Verification 


Options: 

1. ) Direct resources 
toward reduction of p- 
box 

2. ) Reevaluate/Accept 
lower probability 
requirement 

3. ) Reevaluate/Accept a 
less restrictive response 
requirement 

4. ) System/component 
architecture change 
(Redesign) 
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Management of Risk and Resources 
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Model Fidelity 


System level margin for 
uncertainty modeling 

■ Maturity based probability 
requirements 

■ Required epistemic 
uncertainty with maturity 
based burn-down plan 

Model fidelity selection 

■ Maturity based 
requirements 

■ Analysis models with some 
level of uncertainty 
quantification 

■ Matching of model fidelity to 
requirements to save cost 
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Uncertainty Integration 
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Model Fidelity 


Earlier, better, integrated UQ ^ Increased confidence in design 
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Final Remarks 


Confidence can be built into design by using appropriate 
fidelity, integrated uncertainty early in the design cycle. 


An improved understanding of risk helps to guide resource 
investment to save cost in the design as it matures. 

Our experience is that integrated uncertainty quantification 
is not a trivial task!! 
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Questions? 



Backup Slides 



Model Problem Description 


Ballistic Trajectory, Flat Ground, No Drag 
v = initial velocity 
6 = initial angle 
d = strike distance 
x = distance away from initial launch 
h = height 

g = gravitational constant (9.81 m/s 2 ) 


d = 


v 2 sin(2 6) 
g 


h = xtan (6) - 


gx 

2v 2 cos 2 (6 ) 
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Deterministic Solution 


Ballistic Trajectory, Flat Ground, No Drag 
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Aleatory Dispersion of Velocity 


Normal 


Ballistic Trajectory, Flat Ground, No Drag 


300 r 
E" 250 


Velocity ~ N(50,5) m/s 
1000 aleatory samples 
Angle = 45 deg 
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Velocity ~ U(35,65) m/s 
1000 aleatory samples 
Angle = 45 deg 
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CDF of Trajectory Range 
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Epistemic Treatment of Initial Angle 


Velocity treated as aleatory with Uniform distribution 

Initial angle treated epistemically (no distribution modeled, weakest statement) 


300 


E 250 


Ballistic Trajectory, Flat Ground, No Drag 


CDFs of Trajectory Range 


Velocity ~ U(35,65) m/s 
500 aleatory samples 
Angle ~ [20,451 deg 
11 LHS samples 
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Distance from launch point, x (m] 


600 



Epistemic treatment of angle results in a family of CDFs for 
velocity. One for each selection of angle. 
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LHS = Latin Hypercube Sampling 




Probability-box vs Aleatory only 


Taking the Min and Max of the CDFs Results in a P-box for the Response 


Probability Box of Trajectory Range 
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Can’t I just use bigger uncertainties? 


Probability Box of Trajectory Range 



Just increasing an incorrectly classified uncertainty does not yield a 
more conservative result. This is counterintuitive!! 
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Guidance on Uncertainty Informed 

Decisions 


Aleatory dominated predictions (negligible epistemic uncertainty) 
More experimental data and better simulation data 
Will only support the current distribution used 
Not reduce the uncertainty 

Resources directed to robust design 

Epistemic dominated predictions (negligible aleatory uncertainty) 
More experimental data and better simulation data 
Can lead to better understanding 
Should reduce uncertainty 

Resources directed to better understanding the problem 


Mixed aleatory and epistemic predictions 

Use sensitivity analyses to determine resource allocation based on programmatic risk tolerance 

Segregating aleatory and epistemic uncertainty is not a worst-on-worst analysis. 

It is a true representation of both random variation and lack of knowledge. 
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